Mutation testing is a fault-based testing technique that can be used to evaluate the quality of a test suite. The idea is to deliberately seed faults into the original program to create a set of faulty programs, and check whether the test suite can distinguish the behavior of each of the faulty program against the original program. Mutation testing has been used to evaluate software testing at application level, but rarely at the software library level.
Because the size of the software library itself and the number of tests associated with the software library are both very large, a manual approach is not feasible and a suitable tool to automate the evaluation is helpful. One particular challenge that arises from automation is that mutating the software library may affect the behavior of testing tools built upon the software library being tested.